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Abstract.  The  spectral  approach  to  first  passage  time  distributions 
for  Markov  processes  requires  knowledge  of  the  eigenvalues  and 
eigenvectors  of  the  infinitesimal  generator  matrix.  We  demonstrate 
that  in  many  cases  knowledge  of  the  eigenvalues  alone  is  sufficient 
to  compute  the  first  passage  time  distribution. 


1.  Introduction  and  Summary.  Consider  a  continuous  time  Markov 

process  with  state  space  N  =■  (0,1,2,...}.  For  j  <  n  define 

T.  to  be  first  passage  time  from  j  to  (k:  k  ^  n).  Define 
J 

A.  .  to  be  the  n  x  n  matrix  obtained  from  the  first  n  rows  and 
(n) 

columns  of  the  infinitesimal  matrix  A,  i.e.  the  matrix  of  transi¬ 
tion  rates  among  the  states  0,1,..., n-1.  If  A^  is  similar  to 
a  diagonal  matrix  then  spectral  analysis  of  leads  to  the 

representation: 


-*,t 


(1.1) 


Pr(T 


1  n  >  t}  *  i-’  Yie  1 

J  .n  ±ml  i 


In  (1.1),  X ,,..., A  are  the  eigenvalues  of  A,  .  and 

l  n  kn; 

Y,  ,...,Y  depend  on  the  right  and  left  eigenvectors  of  A,  .. 
l  n  kn; 

Convenient  references  for  this  algebraic  approach  are  Cinlar  (1975) , 
Karlin  (1966),  Keilson  (1975)  and  Kerneny  and  Snell  (1960). 

In  general  in  order  to  apply  (1.1)  a  complete  spectral  analysis 
of  A^  is  required.  The  computational  difficulties  in  performing 
this  analysis  limit  the  applicability  of  the  spectral  approach. 


2 


We  consider  a  class  of  Markov  processes  in  which  upward  jumps 


may  only  be  of  size  1  (Ai  i+1>0,  Ai  -0,  j^i+2)  with  no 
restriction  on  downward  jumps.  This  class  includes  birth  and 
death  processes.  For  these  Markov  processes  the  coefficients 
Y^,...,Yn  can  be  computed  from  the  eigenvalues  of  A(n)>  without 
computation  of  right  or  left  eigenvectors.  This  result  should 
simplify  computation  and  render  the  spectral  approach  more  attrac¬ 
tive.  A  summary  of  the  results  now  follows. 

If  X^,...,Xq  are  distinct  then  A(n)  is  similar  to  a  diagonal 
matrix  and  (1.1)  holds.  In  section  2  it  is  shown  that: 


(1.2) 


Pr  (T  >t) 
U,n 


n 

I 

i-1  lj 


yxi 


From  (1.2)  it  follows  that  if  X.  ,...,X  are  real  then  T_ 

in  U,n 

is  the  convolution  of  exponentially  distributed  random  variables 

with  parameters  X^,...,Xn.  If  the  X^'s  are  real  but  not  distinct, 

say  X^,...,Xr  are  distinct  with  multiplicities  m^,...,mr,  then 

Tn  is  distributed  as  the  convolution  of  T(m  ,X  ),  i«l,.,.,r, 

1  1 

where  r(m,X)  is  the  gamma  distribution  with  parameters  m  and  X. 
For  convolutions  of  gammas  we  know  of  no  nice  analogue  of  (1.2)  for 
representing  the  survival  function. 


For  general  j  <  n  ((1.2)  had  j  -  0) ,  let  w- , . . . ,w.  be  the 


eigenvalues  of  Consider  the  case  where  w^,...,Wj  are 

distinct  and  X^,...,Xn  are  distinct.  Define  D  *  {i:X^O{w^, . . .  ,wr)*<J>} 


and  E  ■  {i:w.  fl  (X, , . . .  ,X  }«<$>}.  Then: 
xi  n 


(1.3)  Pr (T  >  t)  *  l  n 

j »“  4^ 


X.  w.  i  -X  t 

l  n  Tif- 1  n  rdb  e 

id)  j*i  j  Ai  JleE  l  i 

J 


The  class  of  processes  under  consideration  can  be  described 
as  birth  and  death  processes  with  multiple  deaths  (but  not  multiple 
births)  allowed.  Birth  and  death  and  related  processes  have  signifi¬ 
cant  application  in  queueing,  reliability,  inventory  and  biological 
models.  Good  references  for  these  applications  are  Cinlar  (1975), 
Gertsbakh  (1984),  Karlin  (1966),  Keilson  (1975),  and  Ross  (1933). 

In  section  3  we  attack  the  problem  of  identifying  y, , . . .  ,y 

l  n 

in  (1.1)  for  a  general  Markov  process. 


2.  Derivation  of  Results.  Consider  a  Markov  process  (X(t),  t>^0} 
with  state  space  N  *  {0,1,2,...}.  The  infinitesimal  generator 
matrix  A  is  defined  by  *  lim  [Pr (X(h)*j |x (0)*i) /h]  for 


h-K) 


i^j,  and  A  -  lim  { [Pr (X(h)-i |X(0)*i)-l] /h} .  We  consider 
h-*0 

processes  which  satisfy: 


(i)  Ai,i+1  *  bi  '  0 

(ii)  A.  .  *  0  for  j  >_  i+2 

(iii)  ”An  <  00  for  all  1 


The  n *  n  matrix  A(n)  is  the  submatrix  of  A  consisting  of 

the  first  n  rows  and  first  n  columns,  i.e.  those  corresponding 

to  states  0,1,..., n-1.  It  is  not  hard  to  show  that  under  (i) , 

(ii)  and  (iii)  above,  A.  *  is  similar  to  a  diagonal  matrix  if  and 

(n; 

only  if  its  eigenvalues  A^,...,A  are  distinct.  Thus  the  spectral 
representation  (1.1)  holds  if  and  only  if  are  distinct. 


Proof  of  (1.2).  In  order  for  state  n  to  be  reached  there  must  be 

at  least  one  transition  from  i  to  i+1  for  i  ■  0,1, . . . ,n-l. 

Thus  Tn  is  stochastically  larger  than  the  convolution  of  n 

exponentially  distributed  random  variables  with  parameters 

bQ,...,bn  p  which  in  turn  is  stochastically  larger  than  M,  the 

maximum  of  n  independent  exponentially  distributed  random  variables 

with  parameters  b_,...,b  .  .  Define  *  Pr(T_  <  t) .  Then: 

u  n-i  U,n  u,n  — 

(  v  Cn-1 

(2.1)  F<:c;  <  Pr(M  <  t)  -  tn  n  b 

0,n-  -  0 

Assume  that  A^,...,An  are  distinct,  and  thus  that  (1.1)  holds. 

Then  is  an  analytic  function  which  by  (2.1)  is  0(tn  1) 

u  j  n 

as  t  -*•  0.  It  is  clear  from  the  Taylor  series  expansion  of  F^C^ 

u  9n 

around  zero  that  FqC^  =  0(tn  ^)  implies: 

(2.2)  F  |  -0  for  k»l,...,n-l. 

dt*  0,n  t-0 

From  (1.1)  and  (2.2): 

n  .  1  for  k  *  0 

(2.3)  l  Y.A*  -  < 

i«l  (0  for  k  *  l,...,n-l  . 

Define  y'  ■  (y. , . . . ,y  ) ,  W  to  be  an  n *  n  matrix  with 
**■  i  n  — 

k-1 

components  ■  A^  ,  i,k  *  1,2, . . .  ,n,  and  £  to  be  an  n  x  1 

vector  with  5^  *  1,  <5^  ■  0,  i«2,...,n.  Rewrite  (2.3)  as: 


A 

+  0(t  )  as  t  -*■  0  . 


(2.4) 


iL>  'iV  A 4 (L  «  {*  i 


Y’W  -  6' 


Now  W  is  the  well-known  Vandermonde  matrix  (Cullen  (1967) 
p.  72).  For  X^,...,\n  distinct  W  is  invertible.  From  (2.4) 
we  see  that  x'  concides  with  the  first  row  of  W~\  thus: 


(2.5) 


yt  *  -  n  r4r- 

J*i  l  j  i 


Substituting  x  int0  (1*1)  yields  (1.2)  Q  . 

Next  note  that  if  are  distinct  then  from  (1.2): 


n  X  a  X  n  K 

l  — -x  r+i  * p(s)  n 

i-ll  j  V  Xi  S  1  i 


(2.6)  iHs)  *  Ee 


where 


(2.7) 


n  r  fX.+s 

l  f "  rbr 

i«lLj*i  lAj 


Now  p(s)-l  is  a  polynomial  of  degree  n-1  with  n  distinct 
roots  X^,...,Xn.  Thus  p(s)-l  =  0  and  it  follows  from  (2.6)  that: 


(2.8) 


n  X 

<Hs)  -  n  t-~- 
i-i  xi+s 


When  X1,...,Xn  are  real  they  must  be  strictly  positive  (since 

(Pr(T_  >t) -*•()),  thus  if  X.  ,...,X  are  real  and  distinct  it 
u  ,n  in 

follows  from  (2.8)  that  T_  is  distributed  as  a  convolution  of 

u,n 

exponentials  with  parameters  X_,...,X  . 

i  n 


’SW 


m 

ww1 


Now  consider  the  case  of  distinct  real  eigenvalues  X^,...,Xr 

with  multiplicity  m^,...,mr.  We  can  obviously  construct  a  sequence 
(k) 

of  vectors  A  converging  to  this  vector  _X  with  each  A 

having  distinct  components.  This  can  be  done  through  a  sequence  of 
(k) 

matrices  A(n)  converging  to  A(n) •  For  the  process  corresponding 
Ck)  (k) 

to  A,  .  the  distribution  of  T'  is  the  convolution  of  n 
— (n)  0,n 

exponentials  and  thus  has  Laplace  transform: 


(2.7) 

Letting  k  -*■  <*>, 
(2.7)  converges  to: 


Ms)  *  n 

K  i-1 


(k) 


LXik)+S. 


(k) 

Trt  converges  in  distribution  to  T_  and 
u,n  0,n 


(2.8) 


ijj(s) 


lim  'f.  (s) 
k-*« 


r  r  * 


m. 


"IxTJ 

i*l L  i 


Since  for  X^,,..,Xr  real  (2.8)  is  the  Laplace  transform  of  a 
convolution  of  r(m^,X^)  random  variables,  we  see  that  Tq  ^  has 
this  distribution. 

We  next  look  at  general  j  <  n  and  prove  (1.3). 


Proof  of  (1.3).  Consider  the  Markov  process  with  the  restrictions 
previously  imposed  and  assume  that  X  ,...,Xn  are  distinct  as  are 
w^,...,Wj  the  eigenvalues  of  A(j)*  Define  D  and  E  as  in  section 
1,  and  D*  *  {l,...,n}-D.  Since  A.  .  ■  0  for  j  >_  i+2,  it  follows 
that 


(2.9) 


T  *  T  *  T 

0,n  U, j  4j,n 


where  *  denotes  convolution.  Applying  (2.8)  to  both  T  and 

”  9  J 

Trt  and  (1.1)  to  T.  we  obtain: 

0,n  j,n 


n  X 


(2.10) 


“  X  +s 
r«l  r 


J  vo 


i-i  Vs 


1  ?  I 

[Vki 

L 

k-1 

l\+sj 

As  some  X^  and  w^  may  coincide  we  divide  both  sides  of 
(2.10)  by  these  common  terms  obtaining: 


(2.11) 


“  X  +s 
reD  r 


w„ 


s  Vk 

teE  k-1 


n 


Multiply  both  sides  of  (2.11)  by  II  (X.+s)  to  obtain: 

j=l  J 


(2.12)  (  n  x  )  n  (x  +s) 

reD  j  eD*  j 


w„ 


teE  Vs' 


1  rk*k  n  (x.+s)  • 

k-1  j^k  J 


Set  s  »  -X^  in  (2.12).  For  ieD  we  obtain: 

X  v 

(2.13)  y.  ■  II  r  J,  -  /  IT  — ^-r—  for  ieD  . 

1  j#i  xr  i  teE  wrAi 

jeD 

For  ieD*,  when  we  set  s  =  -X^  in  (2.12),  the  left  side 
vanishes  while  the  right  side  equals  multiplied  by  a  non-zero 

quantity .  Thus : 


(2.14) 


-  0  for  ieD*  . 


Combining  (2.13)  and  (2.14)  we  obtain  (1.3)  Q  . 


In  the  case  where  either  the  A's  or  w's  are  not  distinct  we 
can  obtain  the  Laplace  transform  of  T  (the  Laplace  transform 

j  »n 

of  T_  divided  by  that  of  T„  . )  but  have  no  convenient  repre- 
O.n  '  0,3 

sentation  for  the  distribution  function. 


3 .  Further  Comments. 

(3.1)  Our  method  worked  well  because  the  first  n-1  derivitives 


of  Fq  ^  were  equal  to  zero  for  the  class  of  processes  considered. 
Applying  the  method  to  a  general  Markov  process  we  obtain  a  represen¬ 
tation  for  F^*"^  but  it  is  not  usually  as  computationally  simple. 

U,n 

Assume  that  A A  are  distinct  which  implies  that  (1.1) 

1  n 

holds.  Note  that: 


(3.1a) 


"k  *  ^  Pr(Ti  n  -  C)  =*  i 

dt  J,n  t*0  i-1 


i  v! 


Note  that  v^-1  and  v  *0  it  re<lu*res  at  least  k+1 
changes  of  state  to  go  from  j  to  (k  n}.  If  v’  =*  ^vo»",,vn-l^ 
is  known  then: 


(3.1b) 


Y'  =  v'W-1  . 


The  problem  reduces  to  computing  v^.  Toward  this  end  consider 
the  lossy  process  with  state  space  0,...,n-l.  The  states  {k  _>  n) 
are  made  absorbing  and  the  Markov  process  restricted  to  0,...,n-l 
steadily  loses  probability  mass  ^j*o  Pr(x(t)=j)<  D  and  thus  called 


'*  .V  ' 


lossy.  Since  Pr(Trt  >  t)  =  Pr(X(t)  <  n-1)  (recall  that  states 

0,n  — 

(k:k  >_  n}  are  absorbing): 


(3.1c) 


A,  .  t 

Pr(T.  >  t)  *  (e  w  1)  ; 
3  »n 


where  1  is  a  column  vector  of  n  l's. 


It  follows  that: 


Pr(T,  >  t)  |  -  (A*  1)  ; 


and  thus: 


(3. Id)  v  =  (-l)k  — -j-  Pr (T  >  t)  |  =  (-l)k(Ak  1)  ; 

k  dtk  J  ’n  t-0  (n)  ~ 


Thus  equals  (-1)  multiplied  by  the  sum  of  the  entries  In 

the  jtk  row  of  the  matrix  Ak^ . 

(3.2)  Keilson  (1975)  proves,  using  different  methodology,  that 
for  a  birth  and  death  process  with  birth  and  death  rates  strictly 
positive,  the  eigenvalues  of  A(n)  are  indeed  distinct  and  real 

and  that  Tq  n  is  distributed  as  the  convolution  of  n  exponentials 
with  parameters  A^,...,^. 

(3.3)  For  the  class  of  processes  considered  we  proved  that  for 


X, ,...,\  real  T_  is  a  convolution  of  exponentials  with  not 
1’  n  0,n  K 

necessarily  distinct  parameters.  Such  a  distribution  belongs  to 


the  class  PF^  (Polya  frequency  function  of  order  n) .  This  class 
enjoys  lots  of  nice  properties  (see  Barlow  and  Proschan  (1975)). 

The  pdf.  is  log  concave  which  leads  to  many  interesting  inequalities 
It  is  also  a  subclass  of  IFR  distributions  thus  all  IFR  inequalities 
are  applicable. 

(3.4)  For  the  class  of  processes  considered  we  do  not  know  of  any 
convenient  to  check  conditions  which  insure  that  the  eigenvalues 
are  distinct  or  real. 
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